Analyzing Learners’ Perception of Indicators in Student-Facing Analytics: A Card Sorting Approach

Abstract

Many studies have explored using different indicators to support students’ self-monitoring. This has motivated the development of student-facing analytics, such as dashboards and chatbots. However, there is a limited understanding of how learners interpret these indicators and act on that information. This study evaluates different indicators from a student perspective by adapting the card sorting technique, which is employed in Human-Centered Design. We chose eight indicators based on different comparative reference frames from the literature to create 16 cards to present both a visual and a text representation per indicator. Qualitative and quantitative data were collected from 21 students of three majors at two Latin American universities. According to the quantitative results, students’ agreement level about the indicators’ interpretability and actionability was relatively low. Nonetheless, the indicators that included temporality were found to be less interpretable but more actionable than those that did not. The analysis indicates that several students would use this information to improve their study habits only if their performance in the course is lower than expected. These findings might be used as a starting point to design student-facing analytics. Also, adapting the card sorting technique could be replicated to understand learners’ use of indicators in other TEL contexts.